Award  Number:  W81XWH-09-2-0089 


AD 


TITLE:  Memorial  Medical  Center  Nursing  Clinical  Simulation  Laboratory 

PRINCIPAL  INVESTIGATOR:  Reed  G.  Williams,  Ph.D. 

CONTRACTING  ORGANIZATION:  Memorial  Health  System 

Springfield,  IL  62781-0001 

REPORT  DATE:  September  2012 

TYPE  OF  REPORT:  Annual 


PREPARED  FOR:  U.S.  Army  Medical  Research  and  Materiel  Command 
Fort  Detrick,  Maryland  21702-5012 


DISTRIBUTION  STATEMENT:  Approved  for  Public  Release; 

Distribution  Unlimited 


The  views,  opinions  and/or  findings  contained  in  this  report  are  those  of  the  author(s)  and 
should  not  be  construed  as  an  official  Department  of  the  Army  position,  policy  or  decision 
unless  so  designated  by  other  documentation. 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and  maintaining  the 
data  needed,  and  completing  and  reviewing  this  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for  reducing 
this  burden  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202- 
4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently 
valid  OMB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 

1.  REPORT  DATE 

September  2012 

2.  REPORT  TYPE 

Annual 

3.  DATES  COVERED 

1  September  201 1  -  31  August  2012 

4.  TITLE  AND  SUBTITLE 

5a.  CONTRACT  NUMBER 

Memorial  Medical  Center  Nursing  Clinical  Simulation  Laboratory 

5b.  GRANT  NUMBER 

W81 XWH-09-2-0089 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

Reed  G.  Williams,  Ph.D. 

5e.  TASK  NUMBER 

E-Mail:  rwilliams@siumed.edu 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Memorial  Health  System 

Springfield,  IL  62781-0001 

8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 

9.  SPONSORING  /  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

U.S.  Army  Medical  Research  and  Materiel  Command 

Fort  Detrick,  Maryland  21702-5012 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 

Approved  for  Public  Release;  Distribution  Unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

The  purpose  of  this  research  is  to  develop  a  program  of  systematic,  brief  training  in  role  appropriate  team  behaviors  covering 
key  communication,  leadership,  and  team  member  behaviors  for  ad  hoc  emergency  medical  care  teams  and  to  determine 
whether  training  in  these  behaviors  results  in  improved  individual  and  team  communication,  leadership,  and  team  member 
performance  in  simulated  emergency  care  situations  regularly  experienced  by  trauma  and  emergency  department  medical  care 
teams.  In  project  year  three  we  have  completed  all  data  collection,  analyzed  and  interpreted  the  data,  written  a  draft  of  the 
primary  manuscript,  presented  the  results  at  annual  national  professional  meetings  for  trauma,  and  emergency  medicine  and 
regional  meetings  of  the  Association  for  American  Medical  Colleges.  We  have  also  provided  didactic  training  for  health  care 
professionals  that  were  not  trained  during  the  original  training.  We  are  in  the  process  of  preparing  the  final  project  report  for 
submission  in  December  2012. 

15.  SUBJECT  TERMS 

Training  of  role  appropriate  communication,  leadership  and  team  behaviors,  ad  hoc  groups  of  emergency  medical  care 
providers,  use  of  simulation  for  training  of  medical  care  providers 

16.  SECURITY  CLASSIFICATION  OF: 


b.  ABSTRACT 

U 


17.  LIMITATION 

18.  NUMBER 

19a.  NAME  OF  RESPONSIBLE  PERSON 

OF  ABSTRACT 

OF  PAGES 

USAMRMC 

19b.  TELEPHONE  NUMBER  (include  area 

UU 

20 

code) 

a.  REPORT 

u 


c.  THIS  PAGE 

u 


uu 


20 


3 


Table  of  Contents 

Page 

Introduction .  4 

Body .  4 

Key  Research  Accomplishments .  10 

Reportable  Outcomes .  11 

Conclusion .  11 

References .  12 


4 


Introduction 

Medical  care  is  increasingly  the  responsibility  of  teams  rather  than  individuals.  This  realization 
has  led  to  increased  team  training  of  medical  professionals  with  a  focus  on  establishing  role 
appropriate  team  behaviors  emphasizing  team  communication,  leadership  and  team  member 
skills.  Emergency  departments  and  trauma  centers  are  unusual  care  settings  in  that 
circumstances  provide  little  time  for  deliberation  and  planning  at  the  time  of  patient  care  due  to 
the  emergency  nature  of  the  cases.  Furthermore,  care  is  provided  by  ad  hoc  teams  where  the 
composition  of  the  group  responsible  for  delivery  of  care  is  not  constant  but  the  roles  and 
responsibilities  needed  are  invariant.  Finally,  members  of  the  team  may  enter  at  various  points 
as  the  case  unfolds.  These  conditions  require  special  team  leadership,  communication  and 
team  member  skills.  The  purpose  of  this  research  is  to  develop  a  program  of  systematic,  brief 
training  in  role  appropriate  team  behaviors  covering  key  communication,  leadership,  and  team 
member  behaviors  for  emergency  medical  care  teams  and  to  determine  whether  training  in 
these  behaviors  will  result  in  improved  targeted  individual  and  ad  hoc  team  communication, 
leadership  and  team  member  behaviors  in  simulated  emergency  care  situations  such  as  those 
regularly  faced  by  trauma  and  emergency  department  medical  care  teams. 

During  the  second  year  we  also  requested  a  no-cost  extension  of  the  project  to  allow  us  time  to 
meet  all  of  the  original  objectives  of  the  project.  This  request  for  an  extension  was  granted  on 
21  -Jul-201 1 .  As  a  result,  the  project  is  now  scheduled  to  be  completed  30  Dec  2012.  In  the 
second  year  annual  report  we  reported  that  we  had  completed  Tasks  1-5  from  the  revised 
milestones  [Task  1 :  Develop  the  training  module  (Milestone  Completion  Date:  31  August  2011), 
Task  2:  Development  of  Simulation  Scenarios  (Milestone  Completion  Date:  31  August  2011), 
Task  3:  Data  collection  and  interpretation  plan  for  analyzing  individual  and  team  performance 
data  during  simulation  scenarios,  Task  4:  Training  interventions  and  the  associated  simulation 
scenarios  pilot  tested.  (Milestone  Completion  Date:  31  August  2011),  Task  5:  Approval  of  the 
project  by  the  local  IRB  and  by  the  USAMRMC  ORP  HRPOj.  The  remainder  of  this  third  year 
annual  report  will  address  the  progress  we  have  made  toward  meeting  the  remaining  revised 
milestones  as  set  down  in  the  approved  no-cost  extension  and  accomplishing  the  original 
objectives  of  the  project.  I  am  happy  to  report  that  the  activities  and  accomplishments  by  project 
staff  reflect  that  the  project  is  now  on  schedule  to  meet  the  milestones  and  complete  the  project 
on  schedule. 

Body 

The  following  provides  a  description  of  project  accomplishments  during  project  year  3  broken 
down  by  the  tasks  and  subtasks  spelled  out  in  the  approved  revised  statement  of  work. 

Tasks  6  and  7  Completed  for  all  Participants.  Task  6.  Training  Intervention  Completed  for 
All  Participants  (Milestone  Completion  Date:  30  September  2011).  Task  7.  Pre,  Post  and 
Three  week  post  simulation  scenarios  administered  to  all  participants  (Milestone 
Completion  Date:  30  September  2012) 


Training  was  completed  for  all  participants.  Participants  included  57  medical  hospital  staff 
(emergency  department  nurses  specially  trained  in  trauma  care,  emergency  medical 
technicians,  respiratory  therapists)  and  physicians  (general  surgery  and  emergency  medicine 
residents)  who  worked  in  trauma  teams  in  the  emergency  department  and  the  Level  One 
trauma  center  at  Memorial  Medical  Center.  Third  to  fifth  year  general  surgery  residents  served 
as  team  leaders.  First,  second,  and,  on  occasion,  third  year  residents  served  as  supporting 
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physician  members  of  the  teams.  These  individuals  were  randomly  assigned  to  teams  for 
simulated  trauma  resuscitations.  This  method  mirrored  the  way  that  ad-hoc  teams  are  formed  to 
provide  emergency  care  in  the  emergency  department  and  trauma  center.  An  Emergency 
Medical  Technician  (EMT)  who  was  not  a  study  participant  was  trained  to  present  the  case  in  a 
standard  manner.  Figure  1  summarizes  the  design  of  the  study.  Table  1  provides  a  description 
of  the  behaviors  targeted  for  training. 

Figure  1.  Study  Design 


The  pretest  simulated  trauma  encounter  (STE)  utilized  high  fidelity  inanimate  computer  driven 
simulators  and  took  approximately  15  minutes  to  complete.  These  pre-intervention  data  were 
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used  to  estimate  baseline  performance  The  STEs  were  video  and  audio  recorded  to  allow 
analysis  of  targeted  individual  and  team  behaviors. 

After  the  baseline  STE,  team  members  participated  in  team  training  designed  to  develop  the 
team  leader  and  member  behaviors  chosen  to  improve  team  communication,  leadership  and 
other  role  appropriate  team  behaviors.  The  training  concluded  with  team  practice  with  a  second 
standardized  patient  STE  which  was  also  used  to  measure  acquisition  of  target  behaviors.  The 
teams  were  debriefed  after  this  post-training  STE  to  help  cement  desired  behaviors  and  to 
correct  undesirable  behaviors. 

Approximately  three  weeks  later,  participants  were  assigned  to  new  teams,  reassembled,  and 
asked  to  participate  in  a  third,  different,  high  fidelity  STE  to  measure  post-intervention  leader 
and  team  member  behaviors.  These  STEs  were  also  video  and  audio  recorded  for  analysis  and 
followed  by  a  team  debriefing  session.  Participants  were  debriefed  after  this  three  week  post 
intervention  retention  STE.  The  STEs  were  randomly  distributed  amongst  the  pre-intervention, 
immediate  post-intervention  and  three  week  post  intervention  conditions  to  control  for  possible 
differences  in  case  difficulty 

Task  8.  Team  performance  During  Pre,  Immediate  Post  and  Three  Week  Post  Intervention 
STEs  were  analyzed  and  interpreted.  (Milestone  Completion  Date:  31  March 
2012) 

Outcomes  and  Measurements.  We  used  Kirkpatrick’s  Hierarchy  (KH)  of  evaluation  (1)  to  guide 
our  choice  of  outcome  measures.  The  primary  outcome  for  the  study  was  changes  in  individual 
and  team  behaviors  (KH  level  3).  These  changes  were  measured  by  comparing  video  records  of 
pre  intervention,  immediate  post  intervention  and  delayed  post-intervention  STEs.  We 
investigated  coded  frequencies  of  trained  behaviors  and  quality  of  those  performances  using  the 
criteria  specified  in  the  training. 

Secondary  outcomes  included: 

•  Percent  who  completed  training  and  views  on  the  learning  experience,  its  organization, 
presentation,  content,  teaching  methods  (KH  1) 

•  Intention  to  apply  knowledge  and  skills  learned  (KH  3) 

•  Other  attitudes  regarding  the  knowledge  and  skills  learned  (KH  3) 

Raters.  A  surgeon  and  a  nurse  who  were  blinded  to  the  study  group  independently  rated  each 
audio/video  record  Raters  blindly  rated  five  of  the  audio  video  records  twice  with  new  video 
record  IDs,  which  allowed  us  to  test  within  rater  agreement.  We  trained  the  raters  by  introducing 
and  talking  about  the  rating  form,  which  we  modified  based  on  their  suggestions.  We  then 
reviewed  a  video  and  rated  it  together,  then  viewed  a  second  video  and  allowed  the  raters  to 
rate  it  and  discuss  their  ratings  and  perceptions. 

Data  Analysis.  We  documented  the  impact  on  the  trainees  by  determining  attitudes  toward  the 
program,  clarity  and  nature  of  expressed  intentions  to  change  personal  work  behavior,  and 
clarity  and  nature  of  expressed  intentions  to  advocate  for  changes  in  work  practices  of  the 
department  and  organization.  Additionally  we  analyzed  differences  in  ad-hoc  individual  and 
team  performance  on  the  STEs  in  each  study  group.  The  unit  of  measurement  was  the 
performance  of  the  team  rather  than  that  of  the  individuals  in  the  group. 

Statistical  Analysis.  Within  rater  agreement  was  determined  using  both  the  relative  ranking  and 
absolute  agreement  models  of  intraclass  correlation  coefficients.  The  first  determines  the 
degree  to  which  the  rankings  of  the  targeted  video  were  similar  on  the  first  and  second 
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evaluations.  The  second  determines  the  degree  to  which  the  absolute  rating  assigned  on  both 
rating  occasions  agreed.  Attitudes  toward  the  course  and  toward  the  targeted  training  behaviors 
were  analyzed  by  determining  the  number  and  percent  of  respondents  choosing  each  response 
option.  Comparisons  between  pre  training,  immediate  post  training  and  three  week  retention 
video  records  of  team  performances  were  compared  using  one-way  analysis  of  variance. 
Selected  target  behaviors  and  combinations  of  target  behaviors  (e.g.  SMARTT  Stepback 
behaviors)  were  also  singled  out  for  analyses.  In  these  cases  the  frequencies  of  occurrences 
and  percent  of  occasions  where  the  behaviors  occurred  were  determined.  All  analyses  were 
performed  using  Statistical  Package  for  the  Social  Sciences  Software  (SPSS)  version  19  (IBM 
SPSS,  Chicago,  Illinois). 


Results 

Table  2  provides  the  results  of  the  within  rater  agreement  analyses  broken  down  by  rater 
(surgeon,  nurse)  and  by  scale.  The  conventionally  accepted  standard  for  reliability  correlations 
is  0.80  and  above  though  this  standard  is  often  not  obtained  when  using  human  raters. 

Our  findings  demonstrated  that  the  nurse  rater  achieved  this  0.80  standard  of  consistency  in 
most  of  the  measurements  included  in  Table  2  regardless  of  the  model  being  used.  The 
surgeon  rater’s  results  were  lower  than  those  of  the  nurse  rater.  For  subsequent  analyses  we 
combined  the  surgeon  and  nurse  rater’s  ratings. 

Table  3  provides  a  summary  of  participant  responses  to  the  post-training  questionnaire 
regarding  their  perceptions  of  the  training.  Forty  five  participants  (79%)  completed  the  post¬ 
training  questionnaire.  Two  participants  did  not  identify  their  medical  role.  Their  responses  are 
included  in  the  Total  columns.  The  first  six  items  established  participants’  perceptions  about  the 
process  and  content  of  the  training.  The  predominant  response  for  all  items  was  “strongly 
agree”  with  the  exception  of  the  item  regarding  “training  being  a  good  use  of  the  respondents’ 
time”.  Within  these  six  questions,  there  were  six  “undecided”  responses.  Four  of  them  were 
responses  to  the  questions  about  being  a  good  use  of  the  respondents’  time.  The  undecided 
responses  were  fairly  evenly  distributed  among  the  three  groups  of  professionals. 

Questions  7  through  1 1  asked  the  participants  for  their  opinions  regarding  the  potential  benefits 
of  the  target  training  behaviors  for  the  quality  and  efficiency  of  patient  care  and  patient 
outcomes.  For  all  five  questions  the  majority  of  participants  strongly  agreed  that  the  training 
had  the  potential  to  improve  patient  safety,  care  efficiency,  team  functioning,  clarity  regarding 
team  leadership,  better  communication,  situation  awareness,  and  mutual  support.  Item  12 
asked  participants  to  indicate  whether  they  intended  to  apply  the  skills  learned  in  their  work 
environments.  Again  the  predominant  response  was  “strongly  agree”. 

Across  these  12  items,  physician  respondents  were  more  positive  about  the  benefits  of  the 
training  and  the  potential  quality  and  care  outcomes  than  were  nurses  and  medical  technicians 
although  all  responded  favorably  with  the  exception  of  the  one  nurse  mentioned  earlier.  The 
final  item  on  the  post-training  questionnaire  asked  the  participants  to  indicate  whether  they  had 
already  used  any  or  all  of  the  trained  skills  in  their  work  environment.  Twenty  six  participants 
(58%)  indicated  they  had  used  some  of  the  trained  skills  already.  Two  indicated  they  had  not 
and  16  said  they  had  not  had  an  opportunity  to  use  the  trained  skills  yet  due  to  the  fact  that  at 
the  time  of  training  the  Level  One  trauma  center  was  at  the  other  teaching  hospital  in 
Springfield. 

Table  4  provides  a  summary  of  the  team  results.  Cells  highlighted  in  light  grey  indicate 
statistically  significant  differences  in  team  and  leader  behaviors  that  suggest  the  training  had 
desired  training  effects.  Cells  highlighted  in  dark  grey  indicate  statistically  significant  differences 
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in  team  behavior  that  indicate  a  significant  reduction  in  desired  team  and  leader  behaviors  at  the 
three  week  retention  endpoint  compared  to  the  immediate  post-training  endpoint. 

As  can  be  seen  in  Table  4,  14  out  of  17  targeted  team  and  leader  behaviors  significantly 
improved  immediately  following  the  training.  One  area  where  team  and  leader  behaviors  did  not 
improve  included  team  member  efforts  to  clarify  ambiguous  orders.  It  is  not  clear  whether  this 
result  reflects  situations  with  ambiguous  orders  where  team  members  did  not  attempt  to  clarify 
the  ambiguity  or  the  finding  reflects  the  lack  of  ambiguous  orders  and  thus  the  opportunity  and 
need  to  seek  clarity.  We  believe  the  former  is  the  more  accurate  explanation.  The  second  area 
where  improvements  were  not  observed  involved  whether  the  team  leader  was  clearly 
identifiable.  As  can  be  seen  in  the  pre-training  simulation  results,  the  judges’  ratings  indicate 
that  the  team  leader  was  easily  identified  prior  to  training.  This  result  may  indicate  the  lack  of  a 
need  for  training  in  this  specific  area.  The  third  area  where  training  effects  were  not  manifested 
in  the  results  involved  team  leader  management  of  noise.  Anecdotally  observers  independently 
commented  on  the  fact  that  there  was  less  extraneous  noise  in  these  STEs  than  is  normally  true 
in  trauma  events.  This  artifact  may  explain  the  lack  of  significant  differences  observed. 
Comparing  the  pre-training  with  three  week  retention  results  indicates  that  seven  areas  had 
lasting  training  effects.  Moreover,  four  of  these  seven  areas  (efficiency,  listened  to  information, 
orders  were  carried  out,  cooperation  and  communication)  are  all  critical  indicators  of  effective 
team  performance.  One  area  where  there  was  a  critical  relapse  in  team  behavior  was  team 
member  confirmation  when  they  completed  tasks.  This  area  requires  additional  attention. 

Table  5  indicates  the  number  of  team  members  who  introduced  themselves  to  the  scribe  upon 
arrival.  As  can  be  seen  by  inspecting  the  table,  the  number  of  team  members  who  announced 
themselves  before  training  was  virtually  zero  and  this  was  true  regardless  of  profession.  The 
rate  of  introductions  improved  immediately  after  training  and  was  maintained  at  a  slightly  lower 
rate  after  three  weeks.  However  the  rate  is  still  not  100%. 

Table  6  summarizes  the  number  of  teams  where  at  least  one  coordinated,  complete  SMARTT 
Stepback  occurred  during  the  training  simulation.  For  this  Table  we  recorded  that  a  SMARTT 
Stepback  occurred  only  if  all  six  components  occurred  at  a  single  time.  As  can  be  seen  the 
frequency  of  SMARTT  Stepbacks  increased  after  training  and  the  rate  was  sustained  after  three 
weeks.  However  the  rate  was  still  less  than  the  desired  100%.  Table  7  indicates  the  number  of 
SMARTT  Stepback  component  behaviors  that  occurred  at  some  point  during  each  simulation,  a 
much  less  restrictive  indication  of  training  success.  In  this  table  a  frequency  of  9  in  a  cell  would 
indicate  that  this  behavior  occurred  one  time  in  that  simulation  event.  As  can  be  seen,  the  two 
raters  appeared  to  use  a  different  rating  strategy  for  the  pretraining.  Their  frequencies  for  the 
immediate  post-training  and  three  weeks  retention  post  training  are  more  similar.  Raters 
recorded  each  element  occurring  close  to  once  per  simulation  event  except  in  the  pre-training 
phase  of  the  study. 


Discussion 

The  in-situ  simulation  that  we  conducted  showed  that  residents  were  not  clear  on  who  was 
leading  resuscitations  and  this  resulted  in  a  shifting  leadership  focus  among  the  residents 
throughout  the  trauma  resuscitation.  Communication  was  fragmented,  incomplete,  and 
frequently  interrupted  thus  requiring  repetition.  The  trauma  bay  was  noisy  with  several  people 
often  talking  at  once.  These  findings  were  confirmed  subsequently  in  observations  of  actual 
trauma  resuscitations  in  the  trauma  bay. 
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Our  training  intervention  was  developed  specifically  to  determine  whether  a  brief  training 
program  would  lead  to  changes  in  team  member  behaviors  that  would  improve  team 
performance  in  these  areas. 

Recently  two  other  studies  have  addressed  similar  training  needs  for  ad  hoc  trauma  team 
members  (2,  3).  Both  of  those  training  interventions  involved  more  training  time  on  the  part  of 
participants,  compared  with  our  study.  Likewise  both  studies  included  changes  in  traditional 
trauma  outcome  parameters  as  well  as  changes  in  team  behaviors  whereas  our  study  focused 
exclusively  on  changes  in  targeted  team  behaviors  manifested  in  simulated  trauma 
resuscitations.  Finally,  both  studies  included  attending  physicians  as  participants  while  our  study 
excluded  attending  physicians.  The  primary  value  that  our  study  adds  to  the  findings  of  these 
two  studies  lies  in  two  areas.  First  and  perhaps  most  important,  our  study  used  expert  judges 
who  were  blinded  to  the  stage  of  training  for  participants.  Our  raters  reviewed  audio-video 
records  of  all  teams  performing  all  trauma  simulations  at  every  stage  of  training.  The  audio¬ 
video  records  were  randomly  ordered  and  the  rater  did  not  know  the  training  stage  for  the 
performing  team.  Second,  our  study  added  the  three  week  post-training  simulation  to  measure 
team  retention  of  targeted  team  and  individual  behaviors.  All  three  studies  demonstrated  that 
relatively  brief  training  episodes  can  lead  to  changes  in  targeted  team  leadership, 
communication  and  coordination  behaviors. 

In  our  study  all  behaviors  were  observed  in  STEs  where  participants  knew  what  behaviors  were 
being  observed  and  recorded  which  may  raise  a  question  about  whether  the  trained  behaviors 
will  persist  in  the  actual  trauma  environment  with  real  patients.  However,  the  results  from  the 
Capella  et  al  (3)  and  Steinemann  et  al  (3)  studies  provide  some  evidence  to  support  transfer  of 
the  training  to  real  trauma  cases.  Our  study  does  strengthen  the  collective  knowledge  from  the 
three  studies  by  blinding  the  raters  to  stage  of  training  and  thus  minimizing  the  possibility  that 
raters’  judgments  are  influenced  by  their  knowledge  of  the  training  stage.  Our  results  are  also 
conservative  in  that  the  retention  outcome  measure  occurs  prior  to  the  impact  of  the  second 
debriefing. 

While  the  results  of  our  study  provide  evidence  that  the  training  is  producing  intended  results,  it 
is  clear  that  the  effect  is  not  robust.  The  targeted  behaviors  are  not  present  in  all  teams  and  the 
sporadic  team  behavioral  characteristics  remind  us  that  the  behaviors  are  likely  to  fade  absent 
practice  and  continued  hospital  leadership  support  in  the  form  of  policies  and  role  modeling. 

For  example  inspection  of  Tables  6  and  7  indicate  that,  at  most,  two  thirds  of  the  team  leaders 
initiated  a  complete  SMARTT  Stepback  event  during  the  simulations  occurring  after  training. 
However  many  of  the  elements  of  a  SMARTT  Stepback  occurred  during  each  post  training 
simulation.  This  suggests  partial  success  in  this  portion  of  the  training  but  certainly  also 
indicates  that  more  training  and  reinforcement  of  these  coordinated  behaviors  will  be  needed  if 
SMARTT  Stepbacks  are  expected  to  occur  on  a  regular  basis. 

All  team  leaders  and  prospective  team  leaders  were  trained  and  were  given  opportunity  to 
practice  these  behaviors  as  a  part  of  this  project.  A  nucleus  group  of  nursing  and  emergency 
department  technicians  also  received  the  training  including  the  practice  simulated  trauma 
resuscitations.  All  other  trauma  team  members  are  receiving  the  didactic  training  but  will  not 
have  the  opportunity  to  practice  using  the  simulated  trauma  resuscitations.  The  trauma 
leadership,  hospital  leadership  and  emergency  medicine  leadership  are  committed  to 
encouraging  all  team  members  to  incorporate  these  behaviors  into  their  individual  and  team 
practices  by  providing  mandatory  training  to  the  remaining  trauma  nurses  and  technicians  and 
to  the  new  general  surgery  and  emergency  department  residents. 
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Finally  we  want  to  echo  the  views  of  the  investigators  in  the  Capella  et  al  study(3)  that  the 
logistics  of  providing  training  to  teams  made  up  of  health  care  providers  from  different 
professions  was  one  of  the  most  challenging  aspects  of  this  project.  Coordinating  the 
scheduling  of  these  training  exercises  involved  working  with  five  different  hospital  and  residency 
program  administrative  bodies.  It  is  no  surprise  to  us  that  there  is  a  great  deal  of  talk  about  the 
desirability  of  multi-professional  team  training  but  the  number  of  examples  of  such  training  in 
hospital  settings  is  limited. 


Key  Research  Accomplishments 

•  Respondents  strongly  agreed  that  the  process  and  content  of  the  training  was 
worthwhile. 

•  Respondents  strongly  agreed  that  the  training  had  the  potential  to  improve  patient 
safety,  care  efficiency,  team  functioning,  clarity  regarding  team  leadership,  better 
communication,  situation  awareness,  and  mutual  support. 

•  Twenty  six  participants  (58%)  indicated  they  had  used  some  of  the  trained  skills  already. 
Two  indicated  they  had  not  and  16  said  they  had  not  had  an  opportunity  to  use  the 
trained  skills  yet  due  to  the  fact  that  at  the  time  of  training  the  Level  One  trauma  center 
was  at  the  other  teaching  hospital  in  Springfield. 

•  Physician  respondents  were  more  positive  about  the  benefits  of  the  training  and  the 
potential  quality  and  care  outcomes  than  were  nurses  and  medical  technicians  although 
all  but  one  responded  favorably. 

•  14  out  of  17  targeted  team  and  leader  behaviors  significantly  improved  immediately 
following  the  training.  One  area  where  team  and  leader  behaviors  did  not  improve 
included  team  member  efforts  to  clarify  ambiguous  orders.  The  second  area  where 
improvements  were  not  observed  involved  whether  the  team  leader  was  clearly 
identifiable.  As  can  be  seen  in  the  pre-training  simulation  results,  the  judges’  ratings 
indicate  that  the  team  leader  was  easily  identified  prior  to  training.  This  result  may 
indicate  the  lack  of  a  need  for  training  in  this  specific  area. 

•  The  third  area  where  training  effects  were  not  manifested  in  the  results  involved  team 
leader  management  of  noise.  Anecdotally  observers  independently  commented  on  the 
fact  that  there  was  less  extraneous  noise  in  these  simulated  trauma  events  than  is 
normally  true  in  trauma  settings.  This  artifact  may  explain  the  lack  of  significant 
differences  observed. 

•  Three  week  retention  results  indicate  that  seven  areas  had  lasting  training  effects. 
Moreover,  four  of  these  seven  areas  (efficiency,  listened  to  information,  orders  were 
carried  out,  cooperation  and  communication)  are  all  critical  indicators  of  effective  team 
performance.  One  area  where  there  was  a  critical  relapse  in  team  behavior  was  team 
member  confirmation  when  they  completed  tasks.  This  area  requires  additional 
attention. 

•  The  frequency  of  complete  SMARTT  Stepback  events  increased  after  training  and  the 
rate  was  sustained  after  three  weeks.  However  the  rate  was  still  less  than  the  desired 
100%.  However  the  frequency  of  SMARTT  Stepback  elements  used  in  isolation  did 
improve. 


Reportable  Outcomes 

Aspects  of  our  findings  have  been  presented  in  poster  form  at  the 
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•  Association  of  American  Medical  Colleges  Central  Group  on  Educational  Affairs  Annual 
Meetings  held  in  Saint  Louis,  Missouri  on  March  29-31, 2012 

•  Emergency  Medicine  Council  of  Emergency  Medicine  Residency  Program  Directors 
Annual  Meetings  held  in  Atlanta,  Georgia  on  April  1  -  April  4,  2012 

•  American  Association  for  the  Surgery  of  Trauma  Annual  Meetings  held  from  September 
12-15,  2012  in  Honolulu,  Hawaii. 

•  A  manuscript  describing  this  research  and  the  findings  has  been  prepared  and  will  be 
submitted  for  presentation  at  the  2013  Annual  Meetings  of  the  Association  for  Surgical 
Education  which  is  a  joint  meeting  with  the  Association  of  Program  Directors  in  Surgery. 
The  manuscript  will  also  be  submitted  for  consideration  to  be  published  in  the  American 
Journal  of  Surgery. 

Conclusion 

While  the  results  of  our  study  provide  evidence  that  the  training  is  producing  intended  results,  it 
is  clear  that  the  effect  is  not  robust.  The  targeted  behaviors  are  not  present  in  all  teams  and  the 
sporadic  team  behavioral  characteristics  remind  us  that  the  behaviors  are  likely  to  fade  absent 
practice  and  continued  hospital  leadership  support  in  the  form  of  policies  and  role  modeling. 

For  example  inspection  of  Tables  6  and  7  indicate  that,  at  most,  two  thirds  of  the  team  leaders 
initiated  a  complete  SMARTT  Stepback  event  during  the  simulations  occurring  after  training. 
However  many  of  the  elements  of  a  SMARTT  Stepback  occurred  during  each  post  training 
simulation.  This  suggests  partial  success  in  this  portion  of  the  training  but  certainly  also 
indicates  that  more  training  and  reinforcement  of  these  coordinated  behaviors  will  be  needed  if 
SMARTT  Stepbacks  are  expected  to  occur  on  a  regular  basis. 

All  team  leaders  and  prospective  team  leaders  were  trained  and  were  given  opportunity  to 
practice  these  behaviors  as  a  part  of  this  project.  A  nucleus  group  of  nursing  and  emergency 
department  technicians  also  received  the  training  including  the  practice  simulated  trauma 
resuscitations.  All  other  trauma  team  members  are  receiving  the  didactic  training  but  will  not 
have  the  opportunity  to  practice  using  the  simulated  trauma  resuscitations.  The  trauma 
leadership,  hospital  leadership  and  emergency  medicine  leadership  are  committed  to 
encouraging  all  team  members  to  incorporate  these  behaviors  into  their  individual  and  team 
practices  by  providing  mandatory  training  to  the  remaining  trauma  nurses  and  technicians  and 
to  the  new  general  surgery  and  emergency  department  residents. 

Finally  we  want  to  echo  the  views  of  the  investigators  in  the  Capella  et  al  study(3)  that  the 
logistics  of  providing  training  to  teams  made  up  of  health  care  providers  from  different 
professions  was  one  of  the  most  challenging  aspects  of  this  project.  Coordinating  the 
scheduling  of  these  training  exercises  involved  working  with  five  different  hospital  and  residency 
program  administrative  bodies.  It  is  no  surprise  to  us  that  there  is  a  great  deal  of  talk  about  the 
desirability  of  multi-professional  team  training  but  the  number  of  examples  of  such  training  in 
hospital  settings  is  limited. 

This  project  is  progressing  according  to  the  approved  schedule  in  the  no-cost  extension.  All 
revised  milestones  have  been  achieved.  All  key  players  from  the  emergency  department  and 
the  trauma  center  played  active  roles  in  the  final  design  and  development  of  the  training  and 
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training  materials.  They  also  played  key,  visible  roles  in  delivering  the  training  and  debriefing  the 
teams  after  simulated  trauma  resuscitations.  Further,  while  outside  the  goals  of  this  contract,  the 
didactic  portion  of  this  training  has  now  been  delivered  to  more  than  90%  of  health  care 
professionals  who  provide  trauma  care  at  Memorial  Medical  Center  with  the  goal  of  training 
them  all.  All  new  general  surgery  and  emergency  medicine  residents  have  also  received  the 
training  and  previously  trained  residents  also  participated  in  these  sessions  as  a  form  of 
refresher  training.  The  Director  of  the  Trauma  Center  has  informed  all  health  care  personnel 
that  his  expectations  are  that  the  trained  procedures  will  be  used  routinely  in  all  trauma  care 
situations  in  this  hospital. 
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Table  I.Team  Behaviors  Targeted  for  Training 


Team  Leader  Behaviors 

Decide  what  needs  to  be  done  and  priorities 

Direct  orders  to  specific  people  by  name  and  or  role  (e.g.  nurse,  respiratory 
technician) 

Issue  short,  clear  orders 

Explicit  transitions  in  team  leadership 

Manage  noise  in  the  trauma  bay 

Manage  workload  to  achieve  team  balance 

Ensure  that  team  members  adhere  to  orders  and  protocol 

Encourage  all  members  to  volunteer  key  information 

Team  Member  Behaviors 

Introduce  self  by  name  and  function  to  the  scribe  and  other  team  members 

Carry  out  orders  issued  by  the  managing  physician  and  standing  orders 
appropriate  to  role 

Communicate  critical  information  to  all  members  of  the  team 

Speak  only  when  necessary  for  patient  care.  Minimize  unnecessary 
noise/distraction/talk 

Listen  to  information  provided  by  other  team  members 

Verbally  confirm  orders  and  completion  of  tasks 

Seek  clarification  regarding  who  is  the  managing  physician  when  ambiguity 
exists 

Communication  Behaviors  -  SMARTT 

Stepback  (Led  by  team  leader) 

Situation  (Patient  description,  injury,  status,  circumstances) 

Management  (Treatment  performed) 

Activity  (What  needs  to  happen  next?) 

Rapidity  (What  needs  to  be  done  first  and  how  quickly?) 

Troubleshoot  (What  may  go  wrong  and  steps  to  correct  or  prevent) 

Talk  to  Me  (Encourage  all  team  members  to  volunteer  key  information,  ask 
clarifying  questions,  etc). 
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Table  2.  Within  Rater  Agreement  for  Five  Repeated  Cases  (Blindly  Rated) 


Within  Rater  Agreement  (Intra-class  correlation  coefficient) 

Scale 

Surgeon  Rater 

Nurse  Rater 

Relative 

Absolute 

Relative 

Absolute 

Ranking 

Agreement 

Ranking 

Agreement 

Professionals  Announced 

0.12 

0.12 

0.84 

0.85 

SMARTT  Step  Back  (Dichotomous  Items) 

0.50 

0.50 

0.84 

0.83 

SMARTT  Step  Back  (Quality  Scale) 

0.60 

0.57 

0.94 

0.94 

Team  Behavior  Scale 

0.74 

0.74 

0.85 

0.78 

Team  Leadership  Scale 

0.54 

0.54 

0.79 

0.77 

Trauma  NoTechs  Scale 

0.60 

0.58 

0.74 

0.75 

Average 

0.52 

0.51 

0.83 

0.82 
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Table  3.  Participant  Views  on  the  learning  experience,  broken  down  by  medical  role  of  respondents  (One  nurse 


respondent  who  responded  strongly  disagree  to  all  items  is  excluded  from  the  table  to  simplify  the  table) 
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Table  4.  Mean  ratings  (and  standard  deviations)  of  targeted  and  trained  ad-hoc  trauma  team  behaviors  during 
simulated  trauma  resuscitations  at  each  stage  in  the  team  training  cycle. 


Category  (Best  Possible 

Score) 

Pretraining 

Immediate 

Post 

Training 

3  Week 

Retention 

Pretraining 

vs 

Immediate 

Post 

Training 

Pretraining 

vs  3  wk 

Retention 

Immediate 

Post 

Training 

vs  3  wk 

Retention 

Mean  (SD) 

Mean  (SD) 

Mean  (SD) 

P 

P 

P 

Organized  (5) 

2.67  (0.66) 

3.67  (0.35) 

3.67  (0.50) 

0.001 

0.001 

NS 

Efficiency  (5) 

3.00  (0.67) 

3.78  (0.36) 

3.78  (0.51) 

0.01 

0.01 

NS 

Supportive  (5) 

3.00  (0.50) 

3.67  (0.25) 

3.44  (0.46) 

0.007 

NS 

NS 

Volunteered  Important 

Information  (5) 

2.61  (0.55) 

3.89  (0.22) 

3.28  (0.69) 

0.00 

NS 

NS 

Listened  to  Information  (5) 

3.06  (0.58) 

3.83  (0.25) 

3.67  (0.25) 

0.001 

0.008 

NS 

Instructions  performed  (5) 

3.17(0.35) 

3.78  (0.26) 

3.61  (0.33) 

0.001 

0.018 

NS 

Clarified  ambiguous 

orders  (5) 

1.89  (0.82) 

3.00  (1.20) 

2.50(1.04) 

NS 

NS 

NS 

Confirmed  completion  of 

tasks  (5) 

1.89  (0.33) 

3.06  (0.68) 

2.39  (0.55) 

0.00 

NS 

0.03 

Leader  clearly  identifiable 

(5) 

3.78  (0.51) 

4.0  (0) 

3.83  (0.35) 

NS 

NS 

NS 

Leader  assigned  tasks  to 

team  members  by  name 

or  role  (5) 

3.11  (0.65) 

3.89(0.22) 

3.28  (0.67) 

0.02 

NS 

NS 
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Category  (Best  Possible 

Score) 

Pretraining 

Immediate 

Post 

Training 

3  Week 

Retention 

Pretraining 

vs 

Immediate 

Post 

Training 

Pretraining 

vs  3  wk 

Retention 

Immediate 

Post 

Training 

vs  3  wk 

Retention 

Mean  (SD) 

Mean  (SD) 

Mean  (SD) 

P 

P 

P 

Leader  managed  noise 

appropriately  (5) 

2.41  (1.20) 

3.50  (0.84) 

3.43  (0.79) 

NS 

NS 

NS 

Leader  coordinated 

communication  and  team 

activity  effectively  (5) 

2.33  (0.75) 

3.44  (0.46) 

2.72  (0.67) 

0.003 

NS 

NS 

Leader  encouraged  team 

members  to  volunteer  key 

information  during 

SMARTT  Step-Back  (5) 

1.50  (0.71) 

3.58  (0.58) 

3.25(1.17) 

0.04 

NS 

NS 

TRAUMA  NOTECHS 

Leadership  (5) 

3.72  (0.36) 

4.67  (0.43) 

4.22  (0.67) 

0.002 

NS 

NS 

Cooperation  (5) 

2.89  (0.65) 

4.61  (0.42) 

3.94  (0.92) 

0.000 

0.01 

NS 

Communication  (5) 

2.56  (0.46) 

4.06  (0.39) 

3.50  (0.97) 

0.000 

0.015 

NS 

Decision  Making  (5) 

3.67  (0.71) 

4.61  (0.70) 

4.17(0.97) 

0.05 

NS 

NS 

Situation  Awareness 

Coping  with  Stress  (5) 

3.67  (0.71) 

4.67  (0.35) 

4.39  (0.70) 

0.005 

0.05 

NS 

Table  5.  Team  Members  Who  Were  Announced  upon  Arrival  Broken  Down  by  Profession  and  by  Reporting 
Rater 

Rater  Chief  ER  Bedside  Junior  Respiratory  Total 

Resident  Technician  Nurse  Resident  Therapist 
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Table  6.  Number  (Percent)  of  Simulations  where  a  SMARTT  Stepback  Occurred  as  a  Coordinated  Event 
(All  components  occurred  at  same  time)  Broken  Down  by  Time  and  by  Rater 


Time 

Rater  One  -  Frequency  (Percent) 

Rater  Two  -  Frequency  (Percent) 

Pre  Training 

0(0) 

0(0) 

Immediate  Post  Training 

3(33) 

6(67) 

Three  Weeks  Post  Training 

5(56) 

6(67) 
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Table  7.  SMARTT  Stepback  Components  that  Occurred  During  the  Simulation  Broken  Down  by  Time  and 
By  Rater  (A  single  performance  which  included  one  instance  of  the  behavior  would  have  a  frequency  of  9  in 
each  cell) 


Time 

Rater 

Situation 

Management 

Activity 

Rapidity 

Troubleshoot 

Talk  to 

Me 

Pre 

Training 

1 

8 

8 

7 

2 

2 

6 

2 

0 

0 

0 

0 

0 

0 

Immediate 

Post 

Training 

1 

9 

9 

9 

8 

8 

9 

2 

6 

6 

6 

6 

5 

5 

Three 

Weeks  Post 

Training 

1 

8 

8 

7 

6 

4 

6 

2 

7 

8 

8 

5 

6 

7 

